The present invention relates to a technique of processing a packet data stream received through a packet-switched network, and particularly, to a technique suitable for real time reproduction processing of a packet data stream representing an audio signal or a video signal.
Recently, real time communication systems using a packet-switched network, such as VoIP (Voice over Internet Protocol), are proposed.
In this kind of real time communication system, in the transmitting end, a media data signal (i.e., an audio signal of, for example, talking voice, or a video signal of, for example, a dynamic image), which requires real time, is sampled at a sampling period, packeted, and transmitted onto a packet-switched network. Thus, the transmitting end transmits sequentially each of packet data constituting a packet data stream that represents the media data signal, onto the packet-switched network in accordance with the above-mentioned sampling period. Then, in the receiving end, each packet data constituting the above-mentioned packet data stream is received through the packet-switched network and processed in the order of reception, so that the media data signal represented by the packet data stream is reproduced.
In the meantime, when communication is performed through a packet-switched network, sometimes a packet data reception period deviates from a packet data transmission period, i.e., a sampling period of a media data signal in the transmitting end, owing to fluctuation of packet data transmission delay times (hereinafter, referred to as transmission fluctuation) on the packet-switched network. In that case, when the receiving end immediately processes packet data received from the packet-switched network, it is impossible to process continuously and sequentially the packet data, which constitute the packet data stream sent from the transmitting end, in accordance with the above-mentioned sampling period. As a result, there arises interruption or collapse of the media data signal.
FIG. 8 is a diagram for explaining a flow of a media data signal in the case where packet data received from a packet-switched network are processed immediately in the receiving end.
In the figure, the reference numeral 701 refers to an input signal as a media data signal inputted into a packet data transmitting apparatus, 702 to a reproduced signal as a media data signal outputted from a packet data receiving apparatus, 703 to a packet data stream transmitted from the packet data transmitting apparatus to a packet-switched network, and 704 to a packet data stream received by the packet data receiving apparatus through the packet-switched network. And, the reference numeral 705 refers to a sampling period T1 for the packet data transmitting apparatus to generate the packet data stream 703 from the input signal 701, and 706 to a packet data transmission period T2 for the packet data transmitting apparatus to transmit sequentially the packet data constituting the packet data stream 703 onto the packet-switched network. Generally, in the case of real time communication, the sampling period T1 705 coincides with the packet data transmission period T2 706. Further, the reference numeral 707 refers to a propagation delay time T3 of a packet data on the packet-switched network.
Now, in FIG. 8, when respective propagation delay times T3 707 of a plurality of successive packet data are equal to each other, as in the case of the first and second packet data in the packet data stream 703, the packet data reception interval T between the plurality of successive packet data coincides with the packet data transmission period T2 706 of the packet data transmitting apparatus. In that case, at a point of time when reproduction processing on a certain packet data has been finished, the packet data receiving apparatus starts reproduction processing on a packet data to be reproduced next to the mentioned packet data. Thus, even when the packet data receiving apparatus processes the packet data immediately after reception from the packet-switched network, a reproduced signal can be obtained without causing interruption or collapse of the reproduced signal.
On the other hand, when transmission fluctuation arises with respect to at least one of a plurality of successive packet data as in the case of the second and third packet data in the packet data stream 703, and consequently, a packet data reception interval T in the mentioned plurality of successive packet data in the packet data receiving apparatus becomes longer than the packet data transmission period T2 706 of the packet data transmitting apparatus, then, sometimes there arises a situation where reproduction processing on a packet data to be reproduced next to a certain packet data is not started at a point of time when reproduction processing on the mentioned certain packet data has been finished. In that case, when the packet data receiving apparatus performs reproduction processing on packet data immediately after reception from the packet-switched network, there arises interruption between a reproduced signal obtained by reproduction of the certain packet data and a reproduced signal obtained by reproduction of the packet data to be reproduced next to that certain packet data.
Further, when there arises transmission fluctuation with respect to at least one of a plurality of successive packet data, as in the case of the third and fourth packet data in the packet data stream 703, and consequently, a packet data reception interval T in the mentioned plurality of successive packet data in the packet data receiving apparatus becomes shorter than the packet data transmission period T2 706 of the packet data transmitting apparatus, then, sometimes there arises a situation where reproduction processing on a packet data to be reproduced next to a certain packet data is started before the reproduction processing on the mentioned certain packet data is finished. In that case, when the packet data receiving apparatus performs reproduction processing on packet data immediately after reception from the packet-switched network, there arises collapse in a reproduced signal obtained by reproducing the certain packet data.
As described above, when a receiving end immediately processes packet data received from a packet-switched network, there arises interruption or collapse of a media data signal owing to transmission fluctuation. Thus, conventionally, the receiving end is provided with a buffer for absorbing transmission fluctuation, and packet data received from the packet-switched network are stored once into the buffer. Then, it is awaited that a predetermined number of packet data are stored in this buffer, and thereafter, the packet data are read sequentially from the buffer to subject them to reproduction processing. Such operation can reproduce the media data signal while preventing occurrence of interruption and collapse owing to transmission fluctuation.
FIG. 9 is a diagram for explaining a flow of a media data signal in the case where packet data received from a packet-switched network are stored once in a buffer and processed. Here, an element having the same function as in FIG. 8 has the same reference numeral.
In the figure, the reference numeral 704a refers to packet data stored in a buffer in the packet data receiving apparatus, 709 to a buffering waiting time T5 extending from storing of the first packet data into the buffer to a start of reproduction processing on packet data, and 710 to a packet data reproduction period T4.
Now, in FIG. 9, when the packet data receiving apparatus receives packet data through the packet-switched network, the packet data are stored once into the buffer. Then, it is awaited that the buffering waiting time T5 709 elapses. And, after the predetermined number of packet data (in this example, two packet data) are stored in this buffer, the packet data are read sequentially from the buffer in accordance with the packet data reproduction period T4 710 to perform reproduction processing. Accordingly, even when transmission fluctuation arises with respect to at least one of a plurality of successive packet data as in the case of the second and third packet data or the third and fourth packet data in the packet data stream 703, and consequently, a packet data reception interval T in the mentioned plurality of successive packet data in the packet data receiving apparatus becomes shorter or longer than the packet data transmission period T2 706 of the packet data transmitting apparatus, the reproduced signal can be obtained without causing interruption or collapse of the reproduced signal.
Here, in providing the receiving end with the buffer for absorbing transmission fluctuation, it is favorable to set the buffer size such that the maximum transmission fluctuation expected on the packet-switched network can be absorbed. However, sometimes, there arises transmission fluctuation exceeding the expected range, on the packet-switched network. When there arises transmission fluctuation that can not be absorbed by the buffer, buffer's underflow (a state in which the buffer is empty and a packet data to be processed next does not arrive at the buffer yet) or overflow (a state in which the buffer is full with packet data and still a packet data arrives one after the other) occurs. Thus conventionally, when buffer's underflow occurs, the reproduction processing on packet data is suspended until the predetermined number of packet data are stored in the buffer. When the predetermined number of packet data are stored in the buffer, then, the reproduction processing on packet data is resumed. When buffer's overflow occurs, the buffer is cleared, and the reproduction processing on packet data is suspended until the predetermined number of packet data are stored in the buffer. When the predetermined number of packet data are stored in the buffer, then, the reproduction processing on packet data is resumed.
FIG. 10 is a diagram for explaining processing in the case where underflow occurs in the buffer provided in the packet data receiving apparatus for absorbing transmission fluctuation, in the example of FIG. 9.
As shown in the figure, when transmission fluctuation exceeding the expected range arises with respect to at least one of a plurality of successive packet data as in the case of the 51st and 52nd packet data in the packet data stream 703, and consequently, a packet data reception interval T in the mentioned plurality of successive packet data in the packet data receiving apparatus becomes extremely longer than the packet data transmission period T2 706 of the packet data transmitting apparatus, then, sometimes there arises a situation (underflow) where, at a point of time when the reproduction processing on a certain packet data is finished, storing of a packet data that should be subjected to the reproduction processing next to the mentioned certain packet data has not stored into the buffer yet. In that case, the reproduction processing on packet data is suspended. Then, it is awaited that the buffering waiting time T5 709 elapses. After the predetermined number of packet data (in this example, two packet data) are stored in this buffer, the reproduction processing on packet data is resumed. As a result, there arises interruption of the buffering waiting time T5 709 in the reproduced signal. Here, the transmission fluctuation that can be absorbed is the larger, the longer the buffering waiting time T5 709 becomes.
FIG. 11 is a diagram for explaining processing in the case where overflow occurs in the buffer provided in the packet data receiving apparatus for absorbing transmission fluctuation, in the example of FIG. 9.
As shown in the figure, when transmission fluctuation exceeding the expected range arises with respect to at least one of a plurality of successive packet data as in the case of 71st and 72nd packet data in the packet data stream 703, and consequently, a packet data reception interval T in the mentioned two successive packet data becomes extremely shorter than the packet data transmission period T2 706 of the packet data transmitting apparatus, then, sometimes there arises a situation (overflow) where, a new packet data arrives at the buffer even though the buffer is full with packet data, in the packet data receiving apparatus. In that case, the buffer is reset once (clearing of the inside of the buffer). Then, it is awaited that the buffering waiting time T5 709 elapses. After the predetermined number of packet data (in this example, two packet data) are stored in this buffer, the reproduction processing on packet data is resumed. As a result, there arises interruption of the buffering waiting time T5 709 in the reproduced signal. Here, the transmission fluctuation that can be absorbed is the larger, the longer the buffering waiting time T5 709 becomes.
As described above, conventionally, a receiving end is provided with a buffer for absorbing transmission fluctuation, and packet data received from a packet-switched network are stored once in this buffer. Then, the packet data are read sequentially from the buffer in accordance with a sampling period, and processed.
Here, conventionally, a precision error between a clock generator (which generates a clock used for generating a packet data stream from an inputted media data signal) in a transmitting end and a clock generator (which generates a clock used for reading packets sequentially from a buffer to perform reproduction processing) in a receiving end is not taken into consideration. An error between those clocks causes difference between a sampling period T1 705 (a packet data transmission period T2 706) in the transmitting end and a packet data reproduction period T4 710. Accordingly, the number of packet data stored in the buffer increases or decreases, finally causing underflow or overflow of the buffer.
Conventionally, buffer underflow and overflow generated by such a cause also have been processed similarly to the buffer underflow and overflow generated by transmission fluctuation exceeding an expected range. As described above, the transmission fluctuation that can be absorbed by the buffer is the larger, the longer the suspension time (which is equal to the buffering waiting time T5 709) of the reproduction processing on packet data in the case of buffer underflow or overflow becomes. Such long interruption of the reproduced signal deteriorates the quality of real time communication.